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SUMMARY 


Five levels of autopilot complexity were flown in a single engine IFR 
simulation for several different IFR terminal operations, A comparison was 
made of the five levels of complexity ranging from no-autopilot to a fully 
coupled lateral and vertical guidance mode to determine the relative benefits 
vs. complexity /cost of state-of-the-art autopilot capability in the IFR 
terminal area. Of the five levels tested, the heading select mode made the 
largest relative difference in decreasing workload and simplifying the 
approach task. It was also found that the largest number of blunders was 
detected with the most highly automated mode. The data also showed that, 
regardless of the autopilot mode, performance during an IFR approach was highly 
dependent on the type of approach being flown. These results indicate that 
automation can be useful when making IFR approaches in a high workload environ- 
ment, but also that some disturbing trends are associated with some of the 
higher levels of automation found in state-of-the-art autopilots. 


INTRODUCTION 


General aviation IFR operations have been increasing rapidly in the past 
few years. This increase is expected to continue and estimates are that the 
number of operations will double within the next 10 years. Along with this 
increasing IFR activity is a corresponding increase in accidents. ^ 

A review of incident and accident data during IFR flights^ shows several 
areas where incidents and/or accidents are most likely to occur. IFR flight 
in the terminal area, for example, during approach and landing, is usually 
associated with one of the highest incident and accident rates in single pilot 
IFR operations. In many of these cases it appears that some level of 

automation might help reduce pilot workload and increase the safety of the 
flight. General aviation pilots, especially those flying single engine 
aircraft, however, have frequently resisted purchasing an autopilot for many 
reasons, (complexity/cost , reliability, pilot acceptance, etc). It is 
suggested that a simple low-cost partial capability autopilot can frequently 
provide sufficient benefits in an IFR environment to justify its use, whereas, 
a complete highly automated autopilot may be undesirable or unaffordable. This 
study compares relative benefits versus complexity/ cost of state-of-the-art 
autopilot capability in the IFR terminal area. 

The paper reports on research comparing various levels of autopilot 
complexity flown in a single engine IFR simulation for several different IFR 
approaches. The analysis reported in the paper represents an overview of the 
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results* Examples are presented to illustrate some of the conclusions* 


ABBREVIATIONS 


ADF 

BC 

GDI 

COM 

DG 

DH 

GS 

HAG 

HC 

HS 

IFR 

ILS 

K 

LOG 

N 

NA 

NAV 

NDB 

OBS 

PIO 

VOR 

WL 

WX 


automatic direction finder 
back course 

course deviation indicator 
communication 
directional gyro 
decision height 
glideslope 

heading select with lateral nav coupler and altitude hold with 
vertical nav coupler 
heading select with lateral nav coupler 
heading select 
instrument flight rules 
instrument landing system 
kilometers 
localizer 
nautical 
no-autopilot 
navigation 

nondirectional radio beacon 
Omni bearing selector 
pilot induced oscillation 
very high frequency omni range 
wing leveler 
weather 


Simulation Facility 


The tests were performed on the NASA Langley general aviation simulator. 
The simulator, flown in the fixed-base mode, was configured and programmed as a 
typical high wing single engine aircraft. Figure 1 shows an outside view of 
the simulator. The cockpit was outfitted with typical basic aircraft instru- 
ments. In addition to these instruments, the following were also included: 
an ADF receiver, two NAV COM systems with corresponding GDI’s, and a complex 
autopilot system. Figure 2 shows an inside view of the cockpit. The 
simulation also incorporated a video out-the-window visual presentation, a 
programmed navigational area encompassing the landing approaches flown, a 
realistic engine and alrstream noise system, and a force feel wheel and 
column control loader. 

The visual out-the-window scene was used for breakout and landing, 
weather permitting. The scene is a video presentation of a map model that 
encompassed a scaled area of approximately 4.4 km (2.4 N miles) by 13.9 km 
(7.5 N miles). ^ Although two airports were located in the scene, all 
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approaches were set up for only one of the airports. However, the second 
airport did play an unplanned part in some of the tests. This will be dis- 
cussed later. Landing and taxiing can be accomplished with this visual 
presentation. 

The programmed navigation area on the computer encompassed the five 
airports used in this study. All the programmed NAV facilities duplicated the 
local real-world NAV environment. All radio aids, magnetic variation, etc, 
were included in the simulation. The simulation did not, however, include 
some of the anomalies associated with the specific real world NAV installation 
(i.e. , scalloping, multiple glideslope paths, etc). 


Method 


Five levels of autopilot automation were tested. The five, in order of 
increasing levels of automation, consisted of: (1) no-autopilot (NA) ; the 

basic aircraft, (2) wing leveler (WL) ; the mode used in this study did not 
have a centering detent on the roll command knob, (3) heading select (HS) ; 
a course selector directional gyro was used in this mode, (4) heading select 
with lateral NAV coupler (HC) ; this mode included lateral guidance for both 
VOR and ILS navigation, and (5) heading select with lateral NAV coupler and 
altitude hold with vertical NAV coupler (HAG); in addition to the previously 
discussed capabilities this mode also included a choice of pitch attitude 
hold, altitude hold, or vertical NAV guidance (i.e., glideslope coupler). 

Five airports and their associated radio NAV aids located in the general 
vicinity of Langley Research Center were programmed and used in this study. 

The types of approaches included two ILS approaches, one VOR approach, one LOG 
BG approach, and one NDB approach » These approaches, and other pertinent 
information, are given in more detail in table I. 

The ceiling and visibility for each data run wens randomly chosen from 
three conditions predefined for each of the five approaches. They were: 

(1) 15.2 m (50 ft) ceiling and 0.8- km (0.5-’mi) minimums for the given approach, 

(2) published minimums for the given approach, or (3) 61 m (200 ft) above ceiling 
and double visibility of published minimums for the given approach. All the runs 
were flown in moderate turbulence (1.2 m/sec (4 ft/sec)) and 20 kt winds from a 
predefined direction. (See table I.) 

Seven subjects were used in the tests: Two NASA test pilots and five IFR 

rated pilots with various levels of IFR and autopilot experience. Each subject 
flew a total of 27 data runs. This included the 25 different combinations of 
five autopilot modes and five different approaches. The extra two runs per 
subject were repeats for replication purposes. The order or presentation was 
randomly determined for each pilot. Simulation sessions were scheduled for 
2-1/2 hours with a 15-minute break halfway through the session. Except for 
one subject, no two sessions were on the same day. Four to five sessions were 
usually required to complete one pilot ^s set of runs. 
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Prior to making any data runs, the subjects were scheduled for a session 
during which they were able to practice all autopilot modes until they were 
satisfied with their performance with the autopilot. The approaches used for 
data (table I) were not used in the practice sessions. 


TABLE I.- APPROACHES 


Airport Runway Approaches Display 


Wind 


Norfolk, VA 

5 

ILS 

Atlanta, GA 

8 

ILS 

Newport News, VA 

25 

LOC/BC 

Franklin, VA 

9 

VOR 

Wakefield, VA 

20 

NDB 



GDI 

091°/20 kt 


GDI 

115° no kt 

folding) 

GDI 

290°/ 20 kt 


GDI 

521° Ho kt 

Fixed 

compass card 

155°/20 kt 


Data Acquisition 


The piloting task consisted of flying the specified approach (table I), 
making the required pilot reports, and performing a side task. The pilot 
reports were specified for the particular approach being flown. The side task 
was a self-pacing velocity/distance/ time problem solved by using a hand held 
E6B type flight computer. For the side task, the subject would, upon his 
request, be given a problem. He would solve the problem, when time was 
available, and report the answer. The radio communication system in the 
simulator was used for this process. The subject was told to perform the side 
task only when it would not interfere with or change the quality of the 
approach being flown. The problems and answers for each run were recorded. 

The pilots were given handouts which included all five approach charts, 
their aircraft location, the initial conditions for each approach, and the 
required reporting points. Table II shows typical initial conditions for one 
of the approaches. 


TABLE II.- INITIAL CONDITIONS 


Newport News LOG BC Rwy 25 


Altitude 
Heading 
Airspeed 
Wind velocity* 
Wind direction 


61 m (2000 ft) 
065 deg 
100 kts 
20 kts 
290 deg 


*From 305 m (1000 ft) to ground wind velocity goes from 20 kts to 10 kts. 
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TABLE !!•- INITIAL CONDITIONS (contVd) 



Newport News LOG BC Rwy 25 

Turbulence 

moderate (1.2 m/sec (4 ft/sec)) 

NX conditions 

day time/ceiling and visibility as 


specified 

Flaps 

0 

NAVI 

110.1 me 

OBS 1 

— 

NAV 2 

116.9 me 

OBS 2 

342 deg 

ADF 

375 kc 

NOTE: Use tear drop 

entry 


The initial conditions positioned the aircraft at a location where a final 
approach and landing clearance would typically be received for that approach. 

At the beginning of each day’s session, the subject was given a practice 
run. Also, prior to each data run, the subjects were given sufficient time to 
review the approach chart, conditions, and procedures. They were then given 
an IFR clearance and reporting points for the approach and the simulation was 
started. The runs were ended after landing and rollout or 10 to 20 seconds 
after initiation of the missed approach. 

The data taken during each approach consisted of flight technical error, 
ground track and profile plots, pilot workload rating and comments, and side 
task results. 


RESULTS AND DISCUSSION 


A review of the data disclosed several events and trends associated with 
pilot performance in flying the various autopilot modes. The following 
analysis is based on pilot comments, pilot ratings, side task results, and 
ground track and profile plots. In analyzing the data, it is necessary to 
consider the interrelationship of several of the above data to fully under- 
stand the results. Results from a single source of data can often be 
misleading. For example, the side task results are dependent not only on task 
difficulty but also on total time required to complete the approach, whereas 
the time required to complete the approach is dependent on the specific 
approach being flown, piloting technique in flying the approach, and the 
difficulty of the approach. Also, the total time to fly the approach may be 
either longer or shorter if the pilot blunders or deviates from the normal 
approach path. 
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The following discussion includes a brief comparison of the five levels 
of autopilot complexity. This is followed by a discussion of the effects of 
the different approaches. Finally, an indepth discussion of trends, as 
related to the various levels of automation, is presented. 


Autopilot Comparison 


Side Tasks .- The side task results, figure 3, in general are representa- 
tive of all the data. This figure shows the average number of problems 
completed per run during all the approaches for all the subjects at each 
level of autopilot complexity. The upper and lower limit bars represent the 
maximum and minimum of the averages of the individual subjects at each level 
of autopilot complexity. Implicit in using a secondary task is the assumption 
that the more difficult the task, the fewer problems completed, hence, the 
higher the workload associated with the primary task. As can be seen by the 
data, the workload tends to decrease (increased secondary task performance) 
as automation level is Increased. Significant, however, is the leveling off 
of the workload for automation levels greater than the HS mode. One 
interpretation of this phenomenon is that beyond the HS mode the subject 
trades off the workload associated with flying the control task for the work- 
load required to monitor the autopilot’s control of the flight task. This 
results in little net difference in primary task workload beyond the HS mode> 

Pilot Workload Ratings .- Figure 4 shows a similar relationship with 
respect to subjective pilot workload ratings. At the end of each run the 
subject rated the primary task on a workload scale of 1 to 7 with 1 designated 
as the easiest and 7 as the hardest. It should be realized that this type of 
rating technique typically produces a relative workload rating of difficulty 
rather than an absolute workload rating. The format of figure 4 is similar 
to that of figure 3, i.e., figure 4 shows the average workload rating per run 
during all the approaches for all the subjects at each level of autopilot 
complexity. The upper and lower limit bars represent the maximum and minimum 
of the averages of the individual subjects at each level of autopilot 
complexity. These results tend to agree with the side task results, i.e., 
increased automation decreases workload. There is also a slight leveling 
off of the workload beyond the HS mode, but it is not as dramatic as in 
the side task data. 


Ground Track Plots .- Figures 5, 6, and 7 show typical pilot control of 
flight ground tracks. The three ground track plots shown are for the NA, TIL, 
and HS autopilot modes. All are for the Atlanta ILS approach and were all 
flown by the same subject. An altitude profile plot is also included in 
figure 5. These figures illustrate the differences in the frequency 
characteristics. The NA mode, figure 5 for example, exhibits two frequencies; 
a high frequency with low damping and a low frequency. As the level of 
automation increases, see figures 6 and 7, the high frequency component 
decreases, in both amplitude and frequency. This results in an apparent 
smoothing of the ground track trace. This smoothing trend with automation was 
characteristic for all the different approaches flown in this study. 



No-autopilot (NA) Mode ,- In the NA mode the pilot flevz the basic aircraft 
without assistance from any autopilot mode. The data, including pilot comments 
and ratings, show this mode to be the most difficult to fly. Typically, the 
biggest problem in flying the NA mode was high workload (as measured by the 
side task and pilot ratings) and less precise flying. Figure 8 shows an 
example of this characteristic for the Newport News LOG BC approach. The 
holding pattern during this run does not conform to a typical pattern. Also, 
the ground track of the NA mode exhibits relatively high frequency and low 
damping characteristics. 

Wing Leveler Mode .- This mode was slightly easier than the 

no-autopilot mode, but some characteristics of the mode were disconcerting. 

Many of the subjects found the WL inputs disturbing when trying to control 
pitch. The control wheel moving in roll interfered with pitch inputs. Also, 
the particular autopilot used in this study did not incorporate a centering 
detent on the roll command knob. This lack of accurate centering frequently 
resulted in the aircraft being in a slight bank with the pilot having to 
continually make inputs to keep wings level, A centering detent is considered 
very desirable, especially when flying in turbulence. An interesting side 
issue is that those subjects not intimately familiar with the WL mode 
commented that it took considerable practice to become comfortable x^rith this 
mode. Even considering all the above, however, all pilots preferred this mode 
to the no-autopilot mode. 

Heading Select (HS) Mode .- The HS mode was considered, by the subjects, 
to be much easier to fly than the ¥L mode. Of the five levels of autopilot 
complexity tested, the HS mode was found to make the largest difference in 
decreasing workload and simplifying the approach task. It was also observed 
that the workload, as measured by the side task, leveled off for the HS, HG, 
and HAG modes (see figure 3). 

Heading Select with Lateral NAV Goupling (HC) Mode .- The next level of 
complexity, the HG mode, was considered somewhat easier than the HS mode but 
not by a large margin. One interesting point, however, is that in this mode 
no comments were made about roll inputs interfering with the pilot *s control 
of pitch. These comments were made in the WL and, to a lesser degree, the 
HS modes. These may be due to the fewer inputs required in these two latter 
modes. 

Heading Select with Lateral NAV Goupling and Altitude Hold with Vertical 

NAV Coupling (HAC) Mode .- The most fully automated mode tested, HAC, as expected, 
was somewhat easier to fly than the HG mode, but again not by a large margin 
over the HC mode. In addition, several problems associated with the HAG mode, 
especially in a high pilot workload environment, became apparent. To a lesser 
degree some of these problems also existed for the HG mode. This will be 
discussed later. 


Approaches 


In addition to the varying levels of difficulty in flying the approach 
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task due to a given level of autopilot automation, the different types of 
approaches were also found to be a factor in difficulty of flying the task. 
This was taken into consideration in analyzing and comparing the autopilot 
data. In general, the data show that the ILS approach (LOG and GS) was the 
easiest to fly* The ILS data included the runs from both the Norfolk and 
Atlanta approaches. 

The Newport News LOG BG and Franklin VOR approaches were about the same 
in overall task difficulty. They were, however, more difficult than the ILS 
approach. Some variability did exist for the two approaches from pilot to 
pilot and from autopilot mode to autopilot mode. It is difficult to make a 
point to point direct comparison of the two approaches due to the difference 
in display sensitivity, the mental gymnastics of reverse sensing, and the 
added task of holding in the LOG BG approach. 

The Wakefield NDB approach was found to be the most difficult by the 
majority of the subjects. This is partly due to the different display used 
in this approach, i.e., the typical ADF relative bearing needle on a fixed 
compass card. This lack of a computed, displayed error for the desired path 
makes the tracking task more difficult. The pilot must continually compute 
error information mentally, using the relative bearing and DG information. 
The differences in difficulty in flying the various approaches can, to a 
large degree, be related to differences in display format, information, and 
sensitivity and to procedures. 


General Trends 


Several disturbing trends were noted as the level of autopilot automation 
was increased. In general, an increased level of automation tends to take 
the pilot out of the aircraft control loop. He becomes a manager of the auto- 
pilot functions. The effects of this change in duty appear to be emphasized 
in the HAG mode. The subjects were more likely to lose track of where they 
were in the approach. It seemed that in monitoring the autopilot they would 
associate instrument readings with the autopilot functions rather than to 
situational awareness. Therefore, if the autopilot functions were either 
set incorrectly or interpreted incorrectly, the subject would frequently 
perform the wrong task, thinking that everything was normal. This v7ould 
frequently lead to an incident or blunder. An example is shown in figure 9 
(Franklin VOR approach, HAG mode). The run began with the autopilot set in 
the heading select mode. After crossing the VOR, a right turn to the 
outbound course was initiated. At this point the autopilot was switched to 
oirnii coupler to intercept and track the outbound course. However, the subject 
had neglected to reset the correct bearing on the GDI. Therefore, the 
autopilot reintercepted and tracked the original bearing of the GDI. 
Eventually, he realized his mistake and set the correct outbound bearing. 

The aircraft then took up a 45^ intercept path to the new bearing. After a 
fair amount of time he still had not intercepted the outbound course but due 
to the time into the approach he decided to make a pseudo procedure turn using 
heading select. At this point in time he also set in the correct inbound 
heading on the GDI. Upon completion of the procedure turn he continued in 
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heading select until the GDI needle came alive. He than selected omni coupler 
and completed the approach without further incident. It is likely this 
incident would not have been detected in the real world. 

Another subject (figure 10, Wakefield NDB approach, HAG mode) made his 
final let down on an outbound heading. He leveled off and made his missed 
approach without ever realizing his mistake. Another interesting facet 
related to this run is the fact that the NDB at Wakefield is located on the 
airport. The missed approach should have been executed when, if in this case, 
the NDB was crossed. In fact several, otherwise nomial, runs were also flown 
at Wakefield in which the missed approach was executed prior to crossing the 
NDB inbound. It seems that the subjects would time their outbound leg and 
use this time, rather than the NDB crossing, to execute their missed approach. 
The 45^ left headwind on the inbound heading was obviously a contributing 
factor in these incidents. This situation implies a lack of positional 
awareness. 

Several other comments about the HAG mode are considered relevant at this 
point. A couple of subjects commented that, while flying the HAG mode, they 
had a tendency, at times, to forget to perform the side task. Another subject 
felt that the altitude hold and glideslope coupler could create a safety issue. 
The pilot can be lulled into a false sense of security or complacency with 
all the automatic features. The problem appears to be almost as if the pilot 
thinks of the autopilot as a copilot and expects it to think for itself. He 
allows himself to become completely engrossed in other tasks once the autopilot 
is set. Hence, he is frequently late in resetting new functions or he may 
become confused as to exactly where he is in the approach and not reset all 
the necessary functions or controls. Still another subject commented that the 
more automated his autopilot the less he trusted it. He stated he had trained 
himself to expect and look for problems of an Insidious nature when using 
complex autopilots. 

The above comments agreed with the relationship of blunders versus 
autopilot automation. The HAG mode encompassed the largest number of 
detectable blunders. 

Remember also that the type of approach was a factor on the prevalence 
of incidents or blunders, the fewest exhibited during the ILS approaches and 
the most during the NDB approach. One notable exception was during an 
Atlanta ILS approach where the subject got into a PIO at the middle marker 
and impacted the ground. (The PIO characteristic of ILS sensitivities 
associated with the middle marker has been observed in independent work at 
NASA LaRG.) The no-autopilot mode was being used for this run. The DH for the 
approach was 61 m (200 ft) above the ground. However, this was only one of two 
blunders for the more than 70 ILS runs flown. The second was when the subject 
executed a missed approach at the outer marker thinking he was at the middle 
marker. The altitude at the outer marker was 853.4 m (2800 ft), whereas at the 
middle marker it would have been 365.8 m (1200 ft). This latter run was flown 
with the HAG mode. 

Several other incidents or blunders not related directly to the autopilot 
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mode are worth mentioning at this point. One subject executed his missed 
approach early for three of the five runs he flew on the Newport News LOC BC 
approach. The wind for this approach was a 45^ right headwind to final 
approach. The subject stated that he intentionally does not use reported 
winds in his missed approach timing. Another problem in LOG BC approach was 
positional disorientation due to reverse sensing on the GDI. One subject 
became so confused he became lost on one run and had to abort. 

The Franklin VOR approach demonstrated similar problems. In several runs, 
for example, the subjects overshot the outbound heading on the approach 
course by a fairly large margin. Also, the procedure turn was, on several 
occasions, considerably larger and out of porportion to the desired path. 

The approach had a 20 knot, 45^ tail wind relative to the final approach 
heading. This tail wind apparently also caused a larger number of missed 
approaches. The subjects would not compensate for the tail wind in their 
approach timing, would descend too slow, and breakout beyond the airport. 

During the Wakefield NDB approach two subjects landed at a second airport 
which just happened to be part of visual model. The second airport was located 
approximately 4.8 km (3 mi) from the destination airport at about 0.8 km (0.5 
mi) to the left side of the desired approach path. Figure 11 shows the 
ground track and profile plots of one of these runs. The location of a 
second airport in the vicinity of the destination airport was not planned as 
part of the experiment. Therefore, the subjects were not previously told 
about the location of the second airport. This incident, having occurred, 
however, emphasizes the problem associated with airports located in the 
vicinity of each other. 


GONGLUDING REMARKS 


A total of 189 IFR approaches were flown on the NASA Langley general 
aviation simulator to compare various levels of automation of autopilot 
systems. Seven IFR rated pilots flew five different airport/approaches with 
five levels of autopilot complexity. 

Of the five levels of autopilot complexity tested, the subjects rated 
each level of added automation to be somewhat easier to fly than the previous 
level, except for one mode. This mode, heading select, was considered to be 
much easier than its next lower level of automation. Also, the data show that 
the heading select mode made the largest difference in decreasing workload 
and simplifying the approach task. The most fully automated mode, which 
included altitude hold and vertical nav coupling, exhibited some disturbing 
aspects, i.e., the largest number of blunders was detected with this mode. 
Also, the side task results showed no decrease in workload from its next 
lower level of automation. 

The data show that the overall quality of performance during an approach 
was highly dependent on the type of approach being flown. The ILS approach, 
localizer, and glideslope were found to be the least difficult. The VOR and 
localizer back course approaches were rated about the same in difficulty, 
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but were considered more difficult than the ILS approach* The NDB approach 
was considered to be the most difficult of those tested* 


The results of this study indicate that automation is desirable when 
making IFR approaches in a high workload environment, but also that some 
disturbing trends are associated with the higher levels of automation as. 
presently implemented in state-of-the-art autopilots. It is believed 
however, that a better man/machine interface could allaviate these problems. 

The data further suggest that the heading select mode may currently be the 
best choice for the IFR approach task when considering both benefits and costs* 
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Figure 3.- Average number of side tasks 
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Figure 4.- Workload ratings. 
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Figure 5.- Atlanta ILS appraoch. No-autopilot mode- (Note: 1 ft = 0.3048 m.) 



DISTANCE (n. ml. ) 


WIND 
,20 kts 


Figure 6.- Groundtraok Atlanta ILS approach. WL autopilot mode. 
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Figure 7.- Groundtrack Atlanta ILS approach. HS autopilot mode. 
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Figure 8.- Groundtrack Newport News LOG BC approach. No-autopilot mode. 
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Figure 9.- Groundtrack Franklin VOR approach. HAC autopilot mode 
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Figure 10.- Wakefield NDB approach. HAC autopilot mode. 
(Note: 1 ft - 0.3048 m. ) 
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Figure 11. - Groundtrack Wakefield NDB approach. HAG autopilot mode. 




